Joint Classification and Regression for Visual Tracking with Fully Convolutional Siamese Networks

Abstract Visual tracking of generic objects is one the fundamental but challenging problems in computer vision. Here, we propose a novel fully convolutional Siamese network to solve visual by directly predicting target bounding box an end-to-end manner. We first reformulate task as two subproblems: classification problem for pixel category prediction and regression object status estimation at this pixel. With decomposition, design simple yet effective architecture based framework, termed SiamCAR, which consists subnetworks: subnetwork feature extraction classification-regression direct prediction. Since proposed framework both proposal- anchor-free, SiamCAR can avoid tedious hyper-parameter tuning anchors, considerably simplifying training. To demonstrate that much simpler achieve superior results, conduct extensive experiments comparisons with state-of-the-art trackers on few benchmarks. Without bells whistles, achieves leading performance real-time speed. Furthermore, ablation study validates various backbone networks, benefit from deeper networks. Code available https://github.com/ohhhyeahhh/SiamCAR .